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Statistical  Inference  for  Bounds  of  Random  Variables 


by  Peter  Cooke 
Statistics  Department, 

University  of  New  South  Wales,  Sydney. 

Summary 

Robson  and  Whitlock  (1964)  considered  point  estimation  and  con- 
fidence limits  for  the  upper  bound  of  a random  variable  when  the  bound 
was  known  to  be  a truncation  point.  However,  their  approach  to  the 
point  estimation  problem  failed  to  produce  an  estimator  with  smaller 
mean  squared  error  than  the  largest  order  statistic  from  a random  sample. 
In  this  paper  we  will  construct  point  estimators  of  the  bounds  of  random 
variables  which  are  substantially  better  estimators  than  the  extreme 
order  statistics  for  many  classes  of  random  variables,  including  those 
whose  distributions  are  truncated  at  one  or  both  ends.  We  will  also 
construct  confidence  limits  and  tests  of  hypotheses  for  bounds.  The 
main  results  are  large  sample  results . 


1.  Introduction 

Suppose  X^,  X2»...,Xn  are  independent  random  variables,  each 

with  absolutely  continuous  cumulative  distribution  function  F(x), 

where  F(x)  £ (0,1)  only  for  x£(c(>,0).  The  interval  (<j> , 0 ) is 

sometimes  referred  to  as  the  support  of  the  distribution  function  F ; 

see,  for  example,  Feller  (1966).  Let  < . . . < Y^  be  the 

order  statistics  based  on  X.,  Xn,...,X  . We  will  construct  a point 

1 2.  n 

estimator  of  0 when  0 is  known  to  be  finite  and  <j)  is  unknown. 
Of  course  the  result  which  follow  also  apply  when  cj>  is  known  and, 


1 


in  particular,  when  <j)  = - 00 , the  only  assumption  required  then 
being  that 

6 2 

/ x dF(x)  < °°  . 

— OO 

We  will  also  construct  confidence  limits  for  6 and  large  sample 
tests  of  hypotheses  about  8 . No  results  for  lower  bounds  will  be 
proved  since  these  can  easily  be  derived  from  the  upper  board  results. 

2.  Point  Estimation  of  0. 


With  no  information  about  the  form  of  F and,  in  particular, 
with  no  information  about  the  shape  of  the  upper  tail  of  f(x)  = F'(x), 
the  statistic  chosen  to  estimate  9 seems  likely  to  be  . In 

an  attempt  to  improve  on  the  estimator  in  the  sense  of  reducing 

its  mean  squared  error  when  9 was  known  to  be  a truncation  point 
of  the  upper  tail  of  f(x),  Robson  and  Whitlock  (1964)  applied  a 
modification  of  Quenoulli's  (1956)  bias  reduction  technique  to  Y^  . 
This  led  to  the  family  of  estimators 


T(k)  = l (-l)1^)  Yn_.  , k = 1,2, .. . ,n-l  , 
i=0 

,(1> 


n 


where  removal  of  the  leading  bias  term  gave  T 

(2) 

next  term  gave  T and  so  on.  However,  it  was  found  that  T 

a n n 


removal  of  the 
(1) 


had  the  same  asymptotic  mean  squared  error  as  Yn  , while  the  mean 

(k) 

squared  error  of  increased  with  k . The  author  has  been 

able  to  show  that  for  the  other  types  of  upper  bound  considered  in 

(k) 

this  paper  the  mean  squared  error  of  T^  also  increases  with  k, 
but  that  T^^  kas  smaHer  asymptotic  mean  squared  error  than  Y^  . 
Thus  we  will  compare  the  mean  squared  error  of  the  estimator  derived 
here  with  that  of 


Y + (Y  - Y ,) 
n n n-1 


2Y  - Y . 
n n-1 


2 


and  of 


The  term  Y - Y , attempts  to  correct  the  bias  in  Y 
n n- 1 r n 

course  does  so  in  a sensible  way  since  Yn  underestimates  0 and 
Yn  - Y^_^  is  nonnegative  with  probability  one  . 


The  random  variable  Y has  distribution  function  F (y) 

n J 


and  mean 


0 0 

E(Y  ) = / ydFn(y)  = 0 - / Fn(y)dy  , 

4»  <!> 


(l) 


on  integrating  by  parts.  Thus  we  can  write 


0 


0 = E(Y  ) + / Fn(y)dy 

<!> 


and  this  suggests  the  estimator 

Yn 

Y + I F"(y)dy 

where  F(y)  is  the  empirical  distribution  function  based  on  the 
order  statistics  Y^,  Y2»...,Y  ; that  is, 

y < Yi 

(2) 


F(y)  = 


Now 


yi  - y < Yi+r  1 = 


i 


y > Y 


n 


Fn(y)dy  - nf  (i)”  (Yi+l  - V 


i=l 


n . n . , n n-1  . n . , , n 

- Y„  - l {©  - (^)  } T±  - Y„  - U (1-  i)  - (1  - 1 Yn.± 

1=1  1=0 

and  so  the  suggested  estimator  is 

n-1  _ . n ...  n 

2Y  - l {(l  - -)  - (l  - — ) }Y  . . 

n .L  1 ' n'  v n ’ 1 n-i 

1=0 

However,  since  we  will  only  investigate  the  case  in  which  n is 
large,  we  will  consider  the  estimator 
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(3) 


A i n-l 

e = 2Y  - (1-e  ) l e_1  Y . 

n n .n  n-i 

i=0 

and  compare  its  performance  with  that  of  . The  asymptotic 

(1)  ~ 

efficiency  of  relative  to  will  be  defined  to  be 

E(6n-6)2 

n » E(T^^-0)2 
n 

When  estimating  <j)  , the  equation 


0 


!(Y  ) - <J)  + / [1-F(y)]ndy 

4> 


suggests  the  estimator 
Y 


n 


^ - / n {1  * E(y)>n  dy  = 2Yj  - l {(l  - )n_  (l  " n H Y± 

Y , l 1 


and  so,  for  large  n,  we  will  consider  the  estimator 

<p  = 2Y.  - (e-1)  [ el.  . 

n 1 . i l 

It  should  be  noted  at  this  stage  that  although  we  will  only 
investigate  properties  of  estimators  based  on  continuous  random 
variables,  in  the  case  of  discrete  variables,  arguments  like  those 

A A 

above  lead  to  similar  estimators  to  and  <j>n  . For  example, 

suppose  a random  variable  with  distribution  function  F can  take 
only  the  integer  values  (j><(j>+l<...<0  with  positive  proba- 
bility. As  above,  suppose  Y^,  Y2>...,Yn  are  the  order  statistics 
based  on  a random  sample  of  size  n from  F . Then 

0 


E(Y  ) = <f>Fn(<|>)  + l y{Fn(y)  - Fn(y-1)}  = 0+1  - j Fn(y) 
n y=cj)+l  y=<)> 


and  so  the  suggested  estimator  of  0 is 

Y 


n„ 


Y 1 + J F”<y)  . 

y-Yi 


4 


Now 


since  the  left  hand  side  has  Y^+^  “ Yi  terms  equal  to 
. n ^ 

(— ) , i = l,2,...,n-l  and  Fn(Y  ) = 1 . Thus  the  estimator  of 


0 is  exactly  as  in  the  continuous  case. 


3.  Asymptotic  Efficiency 


According  to  Gnedenko  (1943),  when  0 is  finite  there  are 
only  two  possible  nondegenerate  limiting  distributions  of  normalized 
values  of  Y^  . In  this  paper  we  will  only  derive  results  for  the 
case 


1/V 


Fn(y)  ~ exp  {-  (|^_  ) } 


as  n 


n 


for  which  the  necessary  and  sufficient  condition  is  that  for 
iim  = c1/V 

v ->  0-  Wy-tf) 


c > 0, 


where  u = F ^(l  - — ) . 

n ' n' 

In  this  section  we  will  discuss  the  asymptotic  efficiency  of 

(1) 

T relative  to  0 when  F satisfies  the  above  condition, 
n n 

First  we  require  expressions  for  means,  variances  and  covariances 
of  order  statistics.  The  following  expressions  are  not  difficult 
to  obtain: 


E(Y  .)  = nC11?1)  / F~ 1 (x) x11-1- 1 ( 1-x)  1dx , 0 < i < n-1  , (4) 

n-i  i 0 - - 

E(Y  |Y  =y)=(n-j-l)  F_1{xF (y)  Jx11-1-'1  (1-x) 1dx,0<j<i<n-i. 

n 1 n j 1 — j —1  q 

(5) 

If 


1/v 


Fn(y)  ~ exp{  - } 


as 


n 


n 


5 


then  F ^(x)  ~ 0 - (0-u  ) (-log  xn)V  as  n -*■  °°  and,  using  (4), 

n 

we  find  that  for  i small 

E(Yn-i)  ~ 6 “ (6-Un)  rm+l)}  as  n **■  “ (6) 

and,  from  (5) , when  i is  small  and  i > j > 0 , 

(0-u  ) 00 


E 


(Yn-i,Yn-j=y)''  9 -f(i=jT  / fz  + (&)  > e"Z  z±~2~ldz  as  n ^ . 

n (?) 

It  is  convenient  to  write 


Cov (Y  ,Y  ) = E(Y  — 0) {Y  -E(Y  )} 

n-i  n j n-i  n— j n— j 


= n(n /)  / {y-E (Y  .)}e(Y  -0|y  =y)¥n~3  (y) {l-F(y) }J  dF(y) 

J (j,  n J n-1  n J 

from  which,  using  (6)  and  (7),  we  obtain 

2 

Cov(Y  Y ) — 2 f E (v+j+l)  ^31  f (*+r')^e~Zz^'~^~^dzdt 

( Vi’  n-jJ~  r(j+i)ra-j)  " r(j+i)  z V ^ 

as  n *>  00  and  finally,  for  i > j > 0 and  i small, 

Cov(Y  Y ) ~ (0-u  ) 2 E (v+j+l)  rr(2v+j+l)  _ I*  (y+i+1 ) t n ->•  00  (8) 

C n-i’  n-j'  V r(J+l)  ^r(v+i+l)  r(i+l)  ‘ as  * w 

Similar  calculations  to  those  above  give 


,2  rr(2v+i+l)  _ r (v+i+1) 
r2(i+i) 


} as  n ->  00 


for  i > 0 and  i small.  Thus  (8)  also  holds  for  i = j . 


(9) 


From  (6) , (8)  and  (9)  we  now  have 


(0-un)"2  E(T^1}-  0)2  - -2^v^y-  r(2v+l) 


as  n -»■  00 


(10) 


We  will  now  find  an  expression  for 
and  (6)  we  obtain 

(0-un)_1  E(0n-0)  - {(l-e-1)"V-2  } r(v+l) 


E(0n-0)  • From  (3) 


as  n -»■  00 


(ID 


6 


and,  using  (3),  (8),  (9)  and  (11), 

(0-un)“2E(en-0)2  ~ 4r(2v+l)+4r2(\H-l)(l-e_1)_V+r(2v+l)(l-e~1)2(l-e“2)“2V“1 


- r2(v+l)(l-e_1)  2V-4r(v+l)(l-e_1) 


V -i  r(2v+i+l) 
.£0e  r(v+i+i) 


+ 2(l-e“V  l 


-1,2  v -i  T(2v+i+l) 


i=l 


r(v+i+i) 


i-l 

I 

j=0 


,-j  r(v+j+i) 

r(j+i) 


^ 6 -n  / > i t \ 3-S 


(12) 


Thus  the  asymptotic  efficiency  of 


,(D 


n 


relative  to  0 


n 


is  given  by 

E(0  -0)2 

ri(V)  = lim  jjz 

n ->  » E(TU  -0) 
n 

which  equals  the  ratio  of  the  expressions  on  the  right  hand  sides  of 
(12)  and  (11).  Some  values  of  n(v)  are  given  to  three  significant 
figures  in  Table  1. 


Table  1:  Values  of  the  asymptotic  efficiency  of  T^^ 

A 

relative  to  0 

n 


V 

1/5 

1/4  1/3  1/2 

1 

n(v) 

1.16 

1.17  1.17  1.08 

.666 

The  value  V = 1 corresponds  to  densities  f (x)  which  are 

A 

truncated  at  0 ; that  is,  0 < f(0)  < 00  . Thus  0^  provides 

a solution  to  the  problem  of  Robson  and  Whitlock,  though  it  is 
slightly  inferior  to  f°r  values  of  v other  than  V = 1 . 

A 

However,  in  the  next  section  we  will  improve  on  ©n  to  produce 
an  estimator  with  smaller  asymptotic  mean  squared  error  than  an 
improved  T^^  for  all  values  of  V considered  in  Table  1.  The 
value  V = \ corresponds  to  densities  f(x)  with  f(0)  equal  to 
zero  or  infinity,  but  with  f T (9)  nonzero  and  finite  and,  in 


7 


general,  V = -^-j-  corresponds  to  a density  which  is  zero  or  infinite 
at  0 and  whose  first  finite,  nonzero  derivative  at  0 is  its 
k*”*1  derivative.  The  derivatives  mentioned  here  are  of  course  all 
left  derivatives  at  0 . 

A 

4.  Improving  on  0^ 

Suppose  now  that  the  value  of  V corresponding  to  the  upper 

tail  of  the  density  f is  known,  though  the  form  of  the  function 
is  unknown.  This  situation  seems  likely  to  occur  in  practical 
problems,  since  for  example  one  might  know  that  he  is  sampling  from  a 
distribution  which  is  truncated  at  0 , so  V = 1 . In  this  case 
it  does  not  seem  to  be  possible  in  general,  by  considering  a function 
of  Yn  alone,  to  improve  on  the  estimator  Yn  either  in  the 
sense  of  reducing  the  order  of  magnitude  of  its  bias  or  of  reducing 
its  mean  squared  error.  For  example,  if  we  consider  estimators 
proportional  to  Y^  , then  the  constant  of  proportionality  which 
minimizes  the  mean  squared  error  of  the  'estimator'  is 

c(a)  = 1 + r (a+l)0-1 (6-u  ) + 0-2  0[ (0-u  )2] 

n n 

which  is  a function  of  0 . 

On  the  other  hand,  the  estimator 

= Y + V~ 1 (Y  -Y  .) 
n n n n-1 

2 

has  bias  of  order  of  magnitude  (0-u^)  compared  with  order  ®-un 

for  T^^  • Using  (6)  and  (8)  we  easily  find 

(0-u  )'2E(?(1)-0)2  ~ T(2V+1)  as  n -»■  «° 
n n 

and 

(0-u  ) 2E(Y’  -0)2  ~ r(2v+l)  as  n -*■  °°  , 

n n 


8 


so  that  bias  reduction  in  is  achieved  at  the  expense  of 

n 

increased  asymptotic  mean  squared  error  except  for  V = 1 where 
there  is  no  change.  The  estimator 

6 = Y + {l-(l-e_1)“V}-1  {Y  -(l-e_1)  £ e_1Y  .} 

n n 1 n n-iJ 

i=0 


also  has  bias  of  order  of  magnitude  (0-u  ) compared  with  order 

n 

A 

0-u  for  0 . Some  values  of 

n n 


nx(v)  = 


the  asymptotic  efficiency  of 


lim 


E(Sn-0r 

E(T(1)-0)2 

n 


;(l) 


n 


relative  to  0 


are  given 


to  three  significant  figures  in  Table  2. 


Table  2:  Values  of  the  asymptotic  efficiency  of  T^^  relative  to  0^. 


V 

1/5 

1/4 

1/3 

1/2 

1 

rij/v) 

.680 

.683 

.689 

.700 

.731 

However,  our  main  aim  is  to  construct  estimators  with  mean  squared 
error  as  small  as  possible.  When  V is  known,  the  estimator  of  0 
of  the  form 


T(1)  = Y + c. (V) (Y  -Y  ,) 

n n 1 n n-1 

with  smallest  mean  squared  error  is  the  one  for  which  c^(v)  = 1/2V 

and,  for  this  choice  of  c^(v),  we  can  show  that 

(6-unr2  Ea^-e)2  - as  . 


The  estimator  of  the  form 


0 

n 


Yn  + c2(V>  (‘-b'1).!  a-H  } 

1=0 


n-1 


with  smallest  mean  squared  error  is  the  one  for  which  C2(v)=a^(v)/a2(v) , 
where 


9 


GO 

ax(v)  = r(v+l)(l-e_1)  l e"1  - T(2vfl) 


i=0 


and 


CO 

,(v)  = r(2v+l){l+(l-e"1)2(l-e"2)"2V_1}-2r(v+l)(l-e“1)  l e"1 


1=0 


+ 2(l-e-1)2  I e-1  ^2^g?}  j^3±^ 


i-1 


i=1  r(v+i+i)  r(  j+i) 


(i) 


The  asymptotic  efficiency  of  Tv  ' relative  to  0 for  the 


n 


n 


above 


choices  of  c^(v)  and  c2(v)  is  given  by 


E (0  -0) 2 

n0(v)  = lim  — 9 

z n ->  00  E(T^iJ-0) 

Some  values  of  ri2(V)  are  given  to  three  significant  figures  in 
Table  3. 


Table  3:  Values  of  c^Cv)  and  the  asymptotic  efficiency  of 

relative  to  0 


n 


1/5 


1/4 


1/3 


1/2 


c2(v) 

n2(v) 


6.42 

.819 


5.11 

.824 


3.80 

.831 


2.49 

.844 


1.18 

.877 


and 


For  practical  purposes  the  simplicity  of  the  estimators  T^ 

T^^  makes  them  attractive.  T^^  could  be  used  when  V is 
n n 


(1) 


unknown,  though  0 is  a much  better  estimator  if  0 happens  to 

n 


be  a truncation  point. 


»<D 


n 


could  be  used  when  V is  known 


since  even  though  it  is  always  inefficient  relative  to  0n  , its 
efficiency  doesn' t drop  below  81%  for  the  values  of  V 
considered  here. 
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5. 


Confidence  Limits  For  0 . 


If 


1/v 


Fn(y)  - exp{-(  |^-)  } as  n 00  , 


n 


then 


lim  pf  -g — — < xl  = 1 - exp(-x^V)  for  x > 0 
n 00  ' Un  * 


and  it  is  not  difficult  to  prove  that  (0  - T^^)/(0-un)  also  has 
a non-degenerate  limiting  distribution  which  is  not  a function  of 
0 . This  leads  us  to  consider  (0-Y  )/f0-T^),  or  equivalently, 

(0-Yn) / (Y^-Y  j.)  as  the  basis  for  constructing  confidence  limits 

for  0 . 


When 


we  find 


lim  P 
n °° 


0-Y 


n 


Y -Y  . 

■ n n-1 


< x 


1/V 

^e-u  J 

r 

n 

X 

-1+x. 

1/V 


0 < x < °° 


(13) 


and  hence,  whatever  the  values  of  0,<j>  or  any  other  parameters 
of  the  distribution  F , 

lim  P[Y  +{(f)"V-l}-1{Y  -Y  }<  0<Y  +{(1-  f)“V-l}_1{Y  -Y  }]=l-a, 
n + a.  n 2 n n-1  - - n 2 n n-1 


lim  P[0  > Y + (a  V-l}  1{Y  -Y  .}]  = 1-ct 
v „ - n n n-  i 

n 0° 

and 

lim  P[0  < Y + { (l-a)-V-l}_1  {Y  -Y  .}]  = 1 - a . 

. . „ - n n n-1 
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When  9 is  a truncation  point  V = 1 and  the  upper  bound 

for  9 in  the  last  statement  equals  Y + (a  ^-1)(Y  -Y  . ) , 

^ n n n- 1 

which  is  Robson  and  Whitlock’s  approximate  upper  confidence  bound 
for  0 . 

6.  Large  Sample  Tests  of  Hypotheses  About  0 . 


Consider  the  hypotheses  Hq  : 0 = 0^  and  : 0 = 0^(<  0^) 

and  the  test  with  rejection  region  for  H : (0  -Y  )/(Y  -Y  1)>{(l-a)  -l} 

0 U n n n- 1 - 

If  we  denote  the  power  function  of  this  test  by  0^(0)  and,  if 


Fn(y)  ~ exp{  - } as  n + °°  , 

n 


1/V 


from  (13)  we  have  lim  0n(0Q)  = 01  ’ whatever  the  values  of  (() 

or  any  other  parameters  of  the  distribution  F . Thus,  when  n 
is  large  the  test  will  have  size  approximately  equal  to  a . Also,  we 
find  the  following  asymptotic  expressions  for  3n(0)  * 


Bn(0) 


1-  (1-a)  { (l-a)-V-l}1/v'(  1_1/v)  [ j 

V n / 

exp[J-{(l-a)'V-l}{-Q^r  }1/V  j 

Bn(0)  ~ a exp  { - I } 

' ny 


l/v.(l-l/v) 

6''-0  ' 1/v-l 

vi/v  xr(i/v) 


as  n 00  for  0<0 


0 ’ 


as  n 00  for 


0 > 0 


0 ' 


(14) 


(15) 


These  expressions  are  not  necessarily  good  representations  of 

Bn(0)  for  0 near  0^  , except  when  V = 1 . However, 

for  0 not  near  0q  and  n large,  | (0q-0)/ (0-un) I will 

be  large,  so  that  both  expressions  are  decreasing  functions  of 

(0-0n)/(0-u  ).  Thus,  if  (0-0rv)/(0-u  ) is  an  increasing  function 
u n u n 
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of  0 , the  above  test  is  a test  of 
: 0 < 0q  with  size  approximately 
decreasing  function  of  0 when  n 


Hq  : 0 > 0O  versus 
a and  power  function  a 
is  large. 


For  a test  of  : 0 = 0^  versus  : 0 = 0^(.  > 8^)  , if 


we  use  the  rejection  region  for  : (O^-Y^) / (Y^-Y^  j)  < ( a V-l)  ^ 

and  denote  the  power  function  by  6 (0),  using  (13)  we  find 
* 

lim  3 (0  ) = a whatever  the  values  of  the  parameters  of  F 
n °°  n 

other  than  8.  We  find  the  following  asymptotic  expressions 

for  3 (8)  J 
n 


B*(8)  ~ a(ofV-l)1/V-(1  1/V)[  Q 


exp 


£-) 

n / 


l/v.(l  l/v)vl/v  lr(1/v) 


as  n 00  for  8 < 0Q, 


(16) 


Bn(8) ~ 1 - (1-a)  exp 


■ 

f e-eQ  u/v  1 

p 

l 0“u  J 

v n * J 

as  n -*■  00  for  0 > 0 


0 


(17) 


As  in  the  previous  case,  except  when  V = 1 , the 
expressions  in  (16)  and  (17)  are  not  necessarily  good  representations 
of  the  power  function  when  0 is  near  0^  . Also,  if 
(0-0o)/(0-un)  is  an  increasing  function  of  0 , the  test  is 
a test  of  Hq  : 0 < 0q  versus  : 0 > 0^  with  size  approximately 

a and  power  function  increasing  in  0 when  n is  large. 
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